Subject determination apparatus that determines whether or not subject is specific subject

ABSTRACT

A subject determination apparatus includes: an image obtaining unit, first and second similarity degree determination units, an information obtaining unit, and a subject determination unit. The second similarity degree determination unit determines whether a similarity degree between a reference image and an image of a candidate region of a specific subject image in one of frame images sequentially obtained by the image obtaining unit is equal to or more than a second threshold value smaller than a first threshold value if the similarity degree is determined by the first similarity degree determination unit to be less than the first threshold value. The information obtaining unit obtains information indicating a similarity degree between the reference image and an image of a region corresponding to the candidate region in another frame image obtained a predetermined number of frames before the one frame image.

BACKGROUND OF THE INVENTION

1. Technical Field

The present invention relates to a subject determination apparatus thatdetermines whether or not a subject is a specific subject, to a subjectdetermination method, and to a recording medium.

2. Description of the Related Art

Heretofore, a technology for performing face detection from frame imagesobtained sequentially based on image capturing has been disclosed, forexample, in a Japanese patent application laid-open publication No.2005-318515.

SUMMARY OF THE INVENTION

A subject determination apparatus according to an embodiment of thepresent invention includes: an image obtaining unit that sequentiallyobtains frame images; a first similarity degree determination unit thatdetermines whether or not a similarity degree between a predeterminedreference image and an image of a candidate region of a specific subjectimage in one of the frame images obtained by the image obtaining unit isequal to or more than a first threshold value; a second similaritydegree determination unit that determines whether or not the similaritydegree is equal to or more than a second threshold value smaller thanthe first threshold value in a case where it is determined by the firstsimilarity degree determination unit that the similarity degree is notequal to or more than the first threshold value; an informationobtaining unit that obtains information related to a similarity degreebetween the predetermined reference image and an image of a region, theregion corresponding to the candidate region in another frame imageobtained a predetermined number of frames before from the one frameimage, in a case where it is determined by the second similarity degreedetermination unit that the similarity degree of the candidate region isequal to or more than the second threshold value; and a subjectdetermination unit that determines whether or not the candidate regionis an image region of the specific subject image based on theinformation obtained by the information obtaining unit.

Moreover, according to an embodiment of the present invention, there isprovided a method of specifying a subject by using a subjectdetermination apparatus, the method including the steps of: sequentiallyobtaining frame images; determining whether or not a similarity degreebetween a predetermined reference image and an image of a candidateregion of a specific subject image in one of the obtained frame imagesis equal to or more than a first threshold value; determining whether ornot the similarity degree between the image of the candidate region andthe predetermined reference image is equal to or more than a secondthreshold value smaller than the first threshold value in a case whereit is determined that the similarity degree is not equal to or more thanthe first threshold value; obtaining information related to a similaritydegree between the predetermined reference image and an image of aregion, the region corresponding to the candidate region in anotherframe image obtained a predetermined number of frames before from theone frame image, in a case where it is determined that the similaritydegree between the image of the candidate region and the predeterminedreference image is equal to or more than the second threshold value; anddetermining whether or not the candidate region is an image region ofthe specific subject image based on the obtained information related tothe similarity degree.

Furthermore, according to an embodiment of the present invention, thereis provided a non-transitory recording medium that records acomputer-readable program that allows a computer to function as: a firstsimilarity degree determination unit that determines whether or not asimilarity degree between a predetermined reference image and an imageof a candidate region of a specific subject image in one of sequentiallyobtained frame images is equal to or more than a first threshold value;a second similarity degree determination unit that determines whether ornot the similarity degree is equal to or more than a second thresholdvalue smaller than the first threshold value in a case where it isdetermined by the first similarity degree determination unit that thesimilarity degree is not equal to or more than the first thresholdvalue; an information obtaining unit that obtains information related toa similarity degree between the predetermined reference image and animage of a region, the region corresponding to the candidate region inanother frame image obtained a predetermined number of frames beforefrom the one frame image, in a case where it is determined by the secondsimilarity degree determination unit that the similarity degree of thecandidate region is equal to or more than the second threshold value;and a subject determination unit that determines whether or not thecandidate region is an image region of the specific subject image basedon the information obtained by the information obtaining unit.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a schematic configuration of an imagecapturing apparatus according to an embodiment to which the presentinvention is applied.

FIG. 2 is a flowchart showing an example of operations related tosubject detection processing by the image capturing apparatus of FIG. 1.

FIG. 3 is a view schematically showing an example of frame imagesrelated to the subject detection processing of FIG. 2.

FIG. 4A is a view schematically showing one of the frame images of FIG.3.

FIG. 4B is a view schematically showing a reduced image of the frameimage of FIG. 3.

FIG. 4C is a view schematically showing a reduced image of the frameimage of FIG. 3.

FIG. 4D is a view schematically showing a reduced image of the frameimage of FIG. 3.

FIG. 4E is a view schematically showing a reduced image of the frameimage of FIG. 3.

FIG. 4F is a view schematically showing a reduced image of the frameimage of FIG. 3.

FIG. 5 is a view schematically showing an example of a discriminationtarget region related to the frame images of FIG. 3.

FIG. 6 is a view schematically showing an example of configurations ofdiscriminators of a similarity degree evaluation value calculation unitrelated to the subject detection processing of FIG. 2.

FIG. 7A is a view schematically showing an example of specific subjectimages related to the subject detection processing of FIG. 2.

FIG. 7B is a view schematically showing an example of the specificsubject images related to the subject detection processing of FIG. 2.

FIG. 7C is a view schematically showing an example of the specificsubject images related to the subject detection processing of FIG. 2.

PREFERRED EMBODIMENTS OF THE INVENTION

A description is made below of specific embodiments of the presentinvention by using the drawings. However, the scope of the invention isnot limited to the illustrated examples.

FIG. 1 is a block diagram showing a schematic configuration of an imagecapturing apparatus 100 to which the present invention is applied.

In the case where it is determined that a similarity degree of acandidate region A of a specific subject image is not equal to or morethan a first threshold value, the image capturing apparatus 100 of thisembodiment determines whether or not the similarity degree concerned isequal to or more than a second threshold value. Moreover, in the casewhere it is determined that the similarity degree of the candidateregion A of the specific subject image is equal to or more than thesecond threshold value, the image capturing apparatus 100 obtainssimilarity degree information related to a similarity degree between apredetermined reference image and an image of a region B, whichcorresponds to the candidate region A of the specific subject image, inanother frame image obtained a predetermined number of frames beforefrom one frame image. Then, while taking the obtained similarity degreeinformation as a reference, the image capturing apparatus 100 determineswhether or not the candidate region A of the specific subject image,which corresponds to the region B in the one frame image, is an imageregion D of the specific subject image.

As shown in FIG. 1, the image capturing apparatus 100 specificallyincludes an imaging section 1, an imaging control section 2, an imagedata generation section 3, a memory 4, an image processing section 5, adiscrimination information setting section 6, a recoding medium controlsection 7, a display control section 8, a display section 9, anoperation input section 10, and a central control section 11.

As an imaging unit, the imaging section 1 captures an image of asubject, and generates frame images F. Specifically, the imaging section1 includes a lens unit 1 a and an electronic imaging unit 1 b.

The lens unit 1 a is composed of a plurality of lenses such as a zoomlens and a focus lens.

The electronic imaging unit 1 b is composed, for example, of an imagesensor such as a charge coupled device (CCD) and a complementarymetal-oxide semiconductor (CMOS), and converts an optical image, whichhas passed through a variety of the lenses of the lens unit 1 a, into atwo-dimensional image signal.

Note that, though not shown, the imaging section 1 may include adiaphragm that adjusts a quantity of light that passes through the lensunit 1 a.

The imaging control section 2 controls an image capturing of the subjectby the imaging section 1. That is to say, though not shown, the imagingcontrol section 2 includes a timing generator, a driver and the like.Then, the imaging control section 2 drives the electronic imaging unit 1b in a scanning manner by the timing generator and the driver, andconverts the optical image into the two-dimensional image signal by theelectronic imaging unit 1 b in every predetermined cycle. Then, theimaging control section 2 reads out the frame images F from an imagingregion of the electronic imaging unit 1 b by every amount of one screen,and outputs the readout frame images F to the image data generationsection 3.

Moreover, the imaging control section 2 performs adjustment/control forimaging conditions of the subject, such as automatic focus processing(AF), automatic exposure processing (AE) and automatic white balance(AWB).

The image data generation section 3 appropriately performs gainadjustment for analog-value signals of the frame images F, which aretransferred thereto from the electronic imaging unit 1 b, for each ofcolor components of R, G and B, thereafter, performs sample holding forthe signals concerned by a sample-and-hold circuit (not shown), andcoverts the signals into digital data by an A/D converter (not shown).Then, the image data generation section 3 performs color processtreatment, which includes pixel interpolation processing andγ-correction processing, for the digital data by a color process circuit(not shown), and thereafter, generates digital-value luminance signals Yand color-difference signals Cb and Cr (YUV data). As described above,the image data generation section 3 generates image data of the frameimages F from such a captured image.

Moreover, the image data generation section 3 performs reductionprocessing for the generated YUV data of the frame images at apredetermined amplification both horizontally and vertically, andgenerates image data for live view display, which have a low resolution(for example, a VGA size, a QVGA size and the like). Specifically, theimage data generation section 3 generates the low-resolution image datafor the live view display from the YUV data of the frame images F atpredetermined timing corresponding to a predetermined display frame rateof a live view image by the display section 9.

Note that the generated image data (YUV data) are DMA-transferredthrough a DMA controller (not shown) to the memory 4 for use as a buffermemory.

The memory 4 is composed, for example, of a dynamic random access memory(DRAM) or the like, and temporarily memorizes data and the like, whichare to be processed by the image data generation section 3, the imageprocessing section 5, the discrimination information setting section 6,the central control section 11, and the like. Specifically, the memory 4temporarily memorizes the image data of the frame images F for the liveview display, of which amount is equivalent to a predetermined number offrames. Here, the image data are data generated by the image datageneration section 3.

Moreover, the memory 4 includes a related information storage unit 4 afor temporarily storing the similarity degree information related tosubject detection processing to be described later.

The related information storage unit 4 a is composed, for example, of aring buffer. The related information storage unit 4 a sequentiallystores, as a data history, the similarity degree information related toa similarity degree between the predetermined reference image and eachof images of candidate regions A (refer to FIG. 7A) of specific subjectimages in the frame images F of which amount is equivalent to thepredetermined number of frames. Here, the similarity degree iscalculated by a similarity degree evaluation value calculation unit 5 cof the image processing section 5. Here, among a plurality of thecandidate regions A . . . in another frame image Fm generated by theimage data generation section 3 predetermined frames (for example, oneframe) before from one frame image Fn, a region, which has coordinatescorresponding to coordinates (for example, coordinates of a center, fourcorners of a rectangular region, and the like) of the candidate regionin the one frame image Fn, becomes the region B (refer to FIG. 7B)corresponding to the candidate region A of the specific subject image.

Moreover, as the similarity degree information, there are mentioned: anevaluation value related to a similarity degree between thepredetermined reference image and an image of the candidate region A(including the region B) of the specific subject image in the otherframe image Fm; a coordinate position of the candidate region A in an XYplane space; a size defined by the number of pixels composing thecandidate region A concerned, and the like; an orientation of thespecific subject image with respect to the XY plane space; the number oftentative candidate regions (described later) detected at substantiallyequal positions in a plurality of reduced image data R . . . of therespective frame images F for use in specifying the candidate region Aconcerned; and the like.

Note that, for example, in the case where a sub-detector (describedlater) for use in detecting a face image as the specific subject imageis provided in response to an orientation of a face, the orientation ofthe specific subject image may be defined in response to a type of thesub-detector concerned.

As described above, the related information storage unit 4 a memorizesthe similarity degree between the predetermined reference image and theimage of the region B corresponding to the candidate region A of thespecific subject image in the other frame image Fm. Here, the similaritydegree is calculated in advance. Moreover, the related informationstorage unit 4 a memorizes the number of tentative candidate regions ofthe specific subject image, which are calculated from the plurality ofreduced images at the substantially equal positions.

The image processing section 5 includes an image obtaining unit 5 a, areduced image generation unit 5 b, the similarity degree evaluationvalue calculation unit 5 c, a first similarity degree determination unit5 d, a second similarity degree determination unit 5 e, a relatedinformation obtaining unit 5 f, a subject determination unit 5 g, and animage region specifying unit 5 h.

The image obtaining unit 5 a sequentially obtains the frame images F.

That is to say, from the memory 4, the image obtaining unit 5 asequentially obtains the image data (for example, data of the luminancesignals Y, and the like) for the live view display of the frame imagesF. Here, the image data is generated at the predetermined timing by theimage data generation section 3.

The reduced image generation unit 5 b generates the reduced image data Rfrom the image data of the frame images F.

That is to say, as an image reducing unit, the reduced image generationunit 5 b sequentially generates reduced images by sequentially reducingthe respective frame images F, which are sequentially obtained by theimage obtaining unit 5 a, at a predetermined ratio. Specifically, basedon the image data of the respective frame images F, the reduced imagegeneration unit 5 b reduces pixels in the respective horizontal (x-axis)and vertical (y-axis) directions in the image data at everypredetermined ratio (for example, by 0.9 time), and thereby sequentiallygenerates the reduced image data R in which the resolution is reducedstep by step (refer to FIG. 4A to FIG. 4F).

Note that the number of generating the reduced image data R (that is,the number of reduction times) is appropriately and arbitrarilychangeable in consideration of a size of luminance data to be inputted,discrimination accuracy of such image regions D (refer to FIG. 7C) ofthe specific subject image, and the like.

The similarity degree evaluation value calculation unit 5 c calculatesan evaluation value related to the similarity degree between thepredetermined reference image and the image of the candidate region A ofthe specific subject image of each of the frame images F.

That is to say, for each of the image data of the respective frameimages F sequentially obtained by the image obtaining unit 5 a and ofthe reduced image data R generated by the reduced image generationsection 5, the similarity degree evaluation value calculation unit 5 cimplements various pieces of image processing, for example, such as facedetection processing, edge detection processing and feature extractionprocessing, and extracts the plurality of candidate regions A (regionscorresponding to detection frames Wb in FIG. 7C) of the specific subjectimage. Then, for example, the similarity degree evaluation valuecalculation unit 5 c calculates an evaluation value related to asimilarity degree between image data of each of the candidate regions Aand image data of the predetermined reference image.

Specifically, the similarity degree evaluation value calculation unit 5c includes a tentative candidate detection unit c1 and a candidateregion specifying unit c2.

As a detection unit, the tentative candidate detection unit c1 detectsthe tentative candidate region of the specific subject image from eachof the plurality of reduced image data R . . . sequentially generated bythe reduced image generation unit 5 b.

That is to say, for example, the tentative candidate detection unit c1generates a plurality of discrimination target regions C with apredetermined size (for example, 24×24 pixels) from the reduced imagedata R in which the resolution of the one frame image Fn is reduced by apredetermined step (refer to FIG. 5). Then, the tentative candidatedetection unit c1 calculates an evaluation value related to a similaritydegree between image data of each of the discrimination target regions Cand the image data of the predetermined reference image, for example, byusing adaboost output calculation.

Specifically, the tentative candidate detection unit c1 includes aplurality of sub-discriminators for calculating the similarity degreebetween each of the discrimination target regions C and thepredetermined reference image and determining whether or not thediscrimination target region C is a face. For example, thesesub-discriminators are provided for each orientation (angle) of the face(for example, for each of a front face, right and left side views, andthe like).

Moreover, these sub-discriminators are defined separately for aplurality (for example, 20) of stages. For example, a predeterminednumber of the sub-discriminators are defined for each of the stages suchthat two thereof are defined for a first stage, five thereof are definedfor a second stage, ten thereof are defined for a third stage, andtwenty thereof are defined for a fourth stage. Note that, for suchsub-discriminators defined at a lower (smaller numeric-value) stage,reliability of determining whether or not the discrimination targetregion C is the face may be set higher.

Then, the tentative candidate detection unit c1 sequentially inputs theimage data of the discrimination target regions C to thesub-discriminators at the respective stages in order of the stages. Thetentative candidate detection unit c1 inputs the image data (“T” in FIG.6), which is determined to be the face in all the sub-discriminators atone stage, to the sub-discriminators at the next stage, and meanwhile,for the image (“F” in FIG. 6) determined not to be the face therein,discontinues subsequent discrimination. The tentative candidatedetection unit c1 calculates the evaluation values for the respectivediscrimination target regions C in accordance with discriminationresults of the sub-discriminators. Specifically, in the case where eachof the discrimination target regions C is determined to be the face inall the sub-discriminators defined at each of the stages in theplurality of stages which define the plurality of sub-discriminators,the tentative candidate detection unit c1 passes the discriminationtarget region C through the stage concerned, and transfers thediscrimination target region C to the next stage. Thereafter, thetentative candidate detection unit c1 calculates a value, which isobtained by adding up the number of sub-discriminators defined in allthe stages through which the discrimination target region C passes, asthe evaluation value. Meanwhile, in the case where the number of stagesthrough which the discrimination target region C passes is “0” (the casewhere the discrimination target region C is not transferred to thesecond stage since the discrimination target region C cannot passthrough the first stage), then for example, the tentative candidatedetection unit c1 calculates a predetermined lowest value such as anegative value as the evaluation value.

Then, the tentative candidate detection unit c1 detects such adiscrimination target region C, in which the evaluation value is largerthan a predetermined value (for example, zero “0”), or a region within apredetermined range, which includes the discrimination target region Cconcerned, as each of the tentative candidate regions (each regioncorresponding to each detection frame Wa in FIG. 7A) of the specificsubject image.

For example, as shown in FIG. 6, the tentative candidate detection unitc1 inputs the image data of the discrimination target regions C to twosub-discriminators of the first stage in a predetermined order, andinputs the image data of the discrimination target regions C, which aredetermined to be the face in both of these two sub-discriminators, tothe sub-discriminators of the second stage as the next stage. Meanwhile,in the case where it is determined that the similarity degree calculatedby the tentative candidate detection unit c1 is less than the presetthreshold value by up to the second sub-discriminator of the firststage, the tentative candidate detection unit c1 discontinues thesubsequent discrimination for the image data concerned, and calculates apredetermined negative value as the evaluation value. Specifically, forexample, the sub-discriminators of the first stage are discriminatorswhich determine an image region with a small luminance change, such as ablue sky and a solid-color background portion. In the case where it isdetermined that an image region is the image region with the smallluminance change, such as the blue sky and the solid-color backgroundportion, at the first stage concerned, the subsequent discrimination asto whether or not the image data is the face is discontinued.

Moreover, in a similar way to the first stage, the tentative candidatedetection unit c1 inputs the image data of the discrimination targetregions C, which are determined to be the face at the first stage, tofive sub-discriminators of the second stage, in a predetermined order,and inputs the image data, in which the similarity degree calculated byfrom the first sub-discriminator up to the fifth sub-discriminator isdetermined to be equal to or more than the preset threshold value, tosub-discriminators of the third stage as a next stage. Meanwhile, in thecase where it is determined that the similarity degree calculated by thetentative candidate detection unit c1 from the first sub-discriminatorof the second stage up to the fifth sub-discriminator thereof is lessthan the threshold value, the tentative candidate detection unit c1discontinues subsequent discrimination for the image data concerned, andcalculates, as the evaluation value, the total number (“2” in the casewhere the discrimination target regions C pass only through the firststage) of the sub-discriminators up to the stage (first stage) onebefore the stage through which the discrimination target regions C havepassed.

Also at the third stage and after, the tentative candidate detectionunit c1 sequentially calculates the similarities in a similar way to theabove. That is to say, an image region having a higher possibility ofbeing determined to be the face is inputted to a sub-discriminator at amore advanced stage.

Note that such a calculation method of the similarity degree by thesimilarity degree evaluation value calculation unit 5 c is merely anexample, and a calculation method according to the present invention isnot limited to this, and is changeable as appropriate. For example, thenumber of the sub-discriminators concerned with the calculation of theevaluation values may be differentiated in response to the orientationof the face (for example, the evaluation value of the front face is sethigh, and the evaluation value in the case where the face is directedsideward or downward is set low, and so on).

Moreover, for example, the image data of the predetermined referenceimage is information memorized in temporarily memorizing means (notshown) of the image processing section 5, and is various pieces ofdiscriminating information for determining whether or not thediscrimination target regions C are the specific subject. Thediscriminating information is various pieces of information fordetermining whether or not the discrimination target regions C, whichare to be inputted to predetermined sub-discriminators, are the specificsubject image (for example, the face and the like). For example, thediscriminating information includes those of the neural net, theadaboost, the support vector machine and the like.

Note that, for example, the discriminating information may beinformation in which a shape of a “human face”, an “animal” or the likeis taken as a reference, in which a color tone such as the fact as towhether the discrimination target regions C have a vivid (fancy) tonewith high brightness and chroma or as to whether a color thereof ispeach is taken as a reference, in which a size such as a ratio of thediscrimination target regions C with respect to an image of the whole ofan angle of view (for example, whether or not a half or more of theimage is occupied) is taken as a reference, or the like.

Moreover, the face detection processing, the edge detection processingand the feature extraction processing are technologies known in public,and accordingly, a detailed description thereof is omitted here.

Based on the tentative candidate regions of the specific subject image,which are detected from the plurality of reduced image data R . . . atthe substantially equal positions of the respective frame images F bythe tentative candidate detection unit c1, the candidate regionspecifying unit c2 as a candidate specifying unit specifies thecandidate region A of the specific subject image in the frame images Fconcerned.

That is to say, in the case where the tentative candidate regions of thespecific subject image are detected from the plurality of reduced imagedata R . . . of the respective frame images F, then based on thecoordinate positions in the XY plane spaces, which are of thepredetermined number of the detected tentative candidate regions of thespecific subject image, on the sizes (number of constituent pixels)thereof, and on the like, the candidate region specifying unit c2integrates these tentative candidate regions with one another, andspecifies the candidate regions A (regions corresponding to thedetection frames Wb in FIG. 7C) of the specific image in the respectiveframe images F. Specifically, the candidate region specifying unit c2performs a predetermined arithmetic operation (for example, weightedaveraging and the like) while taking as references the sizes of theplurality of tentative candidate regions detected from the plurality ofreduced image data R . . . at the substantially equal positions of theXY plane spaces (for example, among the positions, deviation amounts ofthe center coordinate are within a predetermined number of pixels in astate where the sizes of the plurality of reduced images in therespective vertical and horizontal directions are aligned with oneanother), and thereby specifies a size the candidate region A of thespecific subject image, which represents the sizes of the plurality oftentative candidate regions concerned. Moreover, the candidate regionspecifying unit c2 performs a predetermined arithmetic operation whiletaking as references the evaluation values of the plurality of tentativecandidate regions for use in the integration, and thereby calculates anevaluation value of the candidate region A of the specific subjectimage, which represents the evaluation values of the plurality oftentative candidate regions concerned. Furthermore, the candidate regionspecifying unit c2 performs a predetermined arithmetic operation whiletaking as references the coordinate positions of the plurality oftentative candidate regions for use in the integration, and therebyspecifies a position (for example, the center coordinate or the like) ofthe candidate region A of the specific subject image, which representsthe positions of the plurality of tentative candidate regions concerned.

Note that the evaluation value, position, size and the like of thecandidate region A of the specific subject image, which are specified bythe candidate region specifying unit c2, are stored as the similaritydegree information in the related information storage unit 4 a of thememory 4.

The candidate region specifying unit c2 calculates the evaluation valueof the candidate region A of the specific subject image concerned basedon the evaluation values of the plurality of tentative candidateregions. However, by using the adaboost output calculation, for example,the candidate region specifying unit c2 may calculate an evaluationvalue related to a similarity degree between the predetermined referenceimage and the image data of the candidate region A of the specificsubject image, which represents the plurality of tentative candidateregions at the substantially equal positions in the XY plane spaces.

Moreover, the candidate region specifying unit c2 may directly specifythe candidate region A of the specific subject image from the respectiveframe images F, which are obtained by the image obtaining unit 5 a,without specifying the candidate region A of the specific subject imagebased on the plurality of tentative candidate regions detected by thetentative candidate unit c1.

The first similarity degree determination unit 5 d determines whether ornot the evaluation value of the candidate region A of the specificsubject image in the one frame image Fn is equal to or more than thefirst threshold value.

That is to say, as a first similarity degree determination unit, thefirst similarity degree determination unit 5 d determines whether or notan evaluation value related to a similarity degree between the image ofthe candidate region A of the specific subject image in the one frameimage Fn obtained by the image obtaining unit 5 a and a predeterminedreference image serving as a determination criteria of the specificsubject image concerned is equal to or more than the first thresholdvalue. Specifically, the first similarity degree determination unit 5 ddetermines whether or not each evaluation value of the candidate regionA of the specific subject image in the one frame image Fn, which iscalculated by the similarity degree evaluation value calculation unit 5c, is equal to or more than the first threshold value.

Note that, for example, the first threshold value and the secondthreshold value that will be described later may be predetermined valuesinputted based on a predetermined operation for the operation input unit10 by a user, predetermined values preset as defaults, or the like.

The second similarity degree determination unit 5 e determines whetheror not the evaluation value of the candidate region A of the specificsubject image in the one frame image Fn is equal to or more than thesecond threshold value smaller than the first threshold value.

That is to say, in the case where it is determined by the firstsimilarity degree determination unit 5 d that the evaluation value ofthe candidate region A of the specific subject image is not equal to ormore than the first threshold value, the second similarity degreedetermination unit 5 e as a second similarity degree determination unitdetermines whether or not the evaluation value concerned is equal to ormore than the second threshold value smaller than the first thresholdvalue.

The related information obtaining unit 5 f obtains the similarity degreeinformation related to the similarity degree between the predeterminedreference image and the image of the region B in the other frame imageFm.

That is to say, in the case where it is determined by the secondsimilarity degree determination unit 5 e that the evaluation value ofthe image of the candidate region A of the specific subject image isequal to or more than the second threshold value, the relatedinformation obtaining unit 5 f as an information obtaining unit obtainsthe similarity degree information related to the similarity degreebetween the predetermined reference image and the image of the region B,which corresponds to the candidate region A of the specific subjectimage concerned, in the other frame image Fm obtained the predeterminednumber of frames before from the one frame image Fn.

Specifically, the related information obtaining unit 5 f specifies theregion (region B) that has the coordinates corresponding to thecoordinates of the candidate region A of the specific subject image, inwhich it is determined by the second similarity degree determinationunit 5 e that the evaluation value is equal to or more than the secondthreshold value, in the other frame image Fm generated the predeterminednumber of frames (for example, one frame or the like) before from theone frame image Fn (refer to FIG. 7B). Then, the related informationobtaining unit 5 f obtains the similarity degree information of thespecified region B from the related information storage unit 4 a of thememory 4. For example, the related information obtaining unit 5 fobtains, as the similarity degree information, an evaluation valueregarding the similarity degree between the image data of the region Band the image data of the predetermined reference image, the number ofthe tentative candidate regions used for specifying the candidate regionA of the specific subject image, which corresponds to the region Bconcerned, and the like from the related information storage unit 4 a.

Note that the other frame image Fm may have one frame or a plurality offrames. For example, in the case of the plurality of frames, the relatedinformation obtaining unit 5 f obtains similarity degree information ofa region B in a frame image F one frame before therefrom. Then, in thecase where it is determined that the candidate region A of the specificsubject image should not be specified as the image region D of thespecific subject image as a result of a determination by the subjectdetermination unit 5 g to be described later, the related informationobtaining unit 5 f may obtain similarity degree information of a regionB in a frame image F two frames before. As described above, the relatedinformation obtaining unit 5 f may sequentially treat the frame imagesF, each of which is generated one frame before, as processing targets.

Alternatively, the related information obtaining unit 5 f may treat apredetermined number of the frame images F as the processing targets,obtain the relative degree information of the regions B in therespective frame images F, and allow the subject determination unit 5 g,which will be described later, to determine the similarity degreeinformation.

The subject determination unit 5 g determines whether or not thecandidate region A of the specific subject image in the one frame imageFn should be specified as the image region D of the specific subjectimage.

Specifically, as a subject determination unit, the subject determinationunit 5 g determines whether or not the candidate region A is the imageregion D of the specific subject image based on the similarity degreeinformation obtained by the related information obtaining unit 5 f.Specifically, the subject determination unit 5 g determines whether ornot the evaluation value of the region B, which is the similarity degreeinformation obtained by the related information obtaining unit 5 f, isequal to or more than a predetermined determination value (for example,a predetermined value at least larger than the second threshold value,and the like). Moreover, the subject determination unit 5 g determineswhether or not the number of the tentative candidate regions, which isthe similarity degree information obtained by the related informationobtaining unit 5 f, is equal to or more than a predetermineddetermination value (for example, two).

Here, “two” is mentioned as an example of the predetermineddetermination value; however, the predetermined determination value maybe changed to an arbitrary value depending on conditions such as thenumber of the discriminators for use in the face detection, thethreshold values and the like.

Then, in the case where it is determined that the evaluation value ofthe region B is equal to or more than the predetermined determinationvalue, and where it is determined that the number of the tentativecandidate regions is equal to or more than the predetermineddetermination value, the subject determination unit 5 g determines thatthe candidate region A of the specific subject image, which correspondsto the region B in the one frame image Fn, should be specified as theimage region D of the specific subject image.

Note that, in the case where it is determined that either one of theevaluation value of the region B and the number of the tentativecandidate regions is equal to or more than the predetermineddetermination value, the subject determination unit 5 g may determinethat the candidate region A of the specific subject image, whichcorresponds to the region B in the one frame image Fn, should bespecified as the image region D of the specific subject image.

The image region specifying unit 5 h specifies the image region D of thespecific subject image in the one frame image Fn.

Specifically, the image region specifying unit 5 h specifies thecandidate region A (for example, candidate regions A1 to A4 and thelike) of the specific subject image, in which it is determined that theevaluation value is equal to or more than the threshold value by thefirst similarity degree determination unit 5 d, as the image region D ofthe specific subject image (refer to FIG. 7A and FIG. 7C).

Moreover, the image region specifying unit 5 h specifies the candidateregion A of the specific subject image, which corresponds to the regionB in which it is determined that the evaluation value is equal to ormore than the predetermined determination value by the subjectdetermination unit 5 g, as the image region D of the specific subjectimage. Moreover, the image region specifying unit 5 h specifies thecandidate region A of the specific subject image, which corresponds tothe region B in which it is determined that the number of the tentativecandidate regions of the specific subject image is equal to or more thanthe predetermined determination value by the subject determination unit5 g, as the image region D of the specific subject image.

For example, the image region specifying unit 5 h specifies a candidateregion A (for example, the candidate region A5 or the like), in which itis determined by the subject determination unit 5 g that the evaluationvalue is equal to or more than the predetermined determination value andthat the number of the tentative candidate regions of the specificsubject image is equal to or more than the predetermined determinationvalue, as the image region D of the specific subject image (refer toFIG. 7B and FIG. 7C).

Note that such a specifying method of the image region D of the specificsubject image by the image region specifying unit 5 h is merely anexample, and a specifying method according to the embodiment of thepresent invention is not limited to this, and is changeableappropriately and arbitrarily. For example, the image region specifyingunit 5 h may specify a candidate region A of the specific subject image,which corresponds to the region B in which it is determined that theevaluation value is equal to or more than the predetermineddetermination value by the subject determination unit 5 g, as the imageregion D of the specific subject image. Moreover, the image regionspecifying unit 5 h may specify the candidate region A of the specificsubject image, which corresponds to the region B in which it isdetermined that the number of the tentative candidate regions of thespecific subject image is equal to or more than a predetermineddetermination value by the subject determination unit 5 g, as the imageregion D of the specific subject image.

The discrimination information setting section 6 sets discriminationinformation for displaying the specific subject image in adiscriminating manner.

That is to say, when the image region D of the specific subject image isspecified by the image region specifying unit 5 h, the discriminationinformation setting section 6 sets the detection frame Wb, which isdisplayed on the display section 9 while being superimposed on an edgeportion of the image region D of the specific subject image, as thediscrimination information for displaying the specific subject imageconcerned in a discriminating manner.

Note that, though the detection frame Wb, which is displayed while beingsuperimposed on the edge portion of the image region D of the specificsubject image, is illustrated as the discrimination information of thespecific subject image concerned, the detection frame Wb is merely anexample, and discrimination information according to the presentinvention is not limited to this. The discrimination information isappropriately and arbitrarily changeable as long as the informationconcerned displays the specific subject image concerned in adiscriminating manner. Here, the discrimination information includes apredetermined discrimination marker.

The recording medium control section 7 is configured so that a recordingmedium M can be freely attachable/detachable thereto/therefrom, andcontrols readout of data from the recording medium M attached theretoand controls write of data to the recording medium M.

Specifically, the recording medium control section 7 allows therecording medium M to record therein image data for use of recording,which is encoded in accordance with a predetermined compression format(for example, the JPEG format and the like) by an encoding unit (notshown) of the image processing section 5.

Note that, though the recording medium M is composed, for example, of anon-volatile memory (flash memory) and the like, the non-volatile memoryis merely an example, and a recording medium according to the presentinvention is not limited to this, and is changeable appropriately andarbitrarily.

The display control section 8 performs control to read out the imagedata for use of display, which is temporarily memorized in the memory 4,and to allow the display section 9 to display the image data concernedthereon.

Specifically, the display control section 8 includes a video randomaccess memory (VRAM), a VRAM controller, a digital video encoder, andthe like. Then, the digital video encoder periodically reads out theluminance signals Y and the color-difference signals Cb and Cr, whichare read out from the memory 4 and memorized in the VRAM (not shown)under the control of the central control section 11, through the VRAMcontroller from the VRAM, generates video signals based on these data,and outputs the generated video signals to the display section 9.

For example, the display section 9 is a liquid crystal display panel. Ona display screen, the display section 9 displays the image, which iscaptured by the electronic imaging unit 1 b, and the like based on thevideo signals from the video control section 8. Specifically, thedisplay section 9 displays the live view image while sequentiallyupdating the plurality of frame images F . . . at a predetermineddisplay frame rate. Here, the frame images F . . . are captured by theimaging section 1 and the imaging control section 2 and generated by theimage data generation section 3 in a still image capturing mode or amoving picture capturing mode. Moreover, the display section 9 displaysa “rec view” image of an image to be recorded as a still image, anddisplays an image that is being recorded as a moving picture.

The operation input section 10 is a section for performing suchpredetermined operations of the image capturing apparatus 100 concerned.Specifically, the operation input unit 10 includes a shutter button (notshown) for inputting an image capturing instruction of the subject, aselection deciding button (not shown) for inputting selectioninstructions for the image capturing mode, functions and the like, azoom button (not shown) for inputting an adjustment instruction of azoom amount, and the like, and outputs predetermined operation signalsto the central control section 11 in response to operations of thesebuttons by the user.

The central control section 11 is a section that controls the respectivesections of the image capturing apparatus 100. Though not shown,specifically, the central control section 11 includes a centralprocessing unit (CPU), a random access memory (RAM), a read only memory(ROM), and the like, and performs a variety of control operations inaccordance with a variety of processing programs (not shown) for theimage capturing apparatus 100.

Next, a description is made of the subject detection processing by theimage capturing apparatus 100 with reference to FIG. 2 to FIG. 7.

FIG. 2 is a flowchart showing an example of the operations related tothe subject detection processing. Moreover, FIG. 3 is a viewschematically showing an example of images related to the subjectdetection processing. FIG. 4A is a view schematically showing the frameimage F, and FIG. 4B to FIG. 4F are views schematically showing thereduced images of the frame image F. FIG. 5 is a view schematicallyshowing an example of the discrimination target region C related to theframe image F. FIG. 6 is a view schematically showing an example ofconfigurations of the sub-discriminators of the similarity degreeevaluation value calculation unit 5 c related to the subject detectionprocessing. FIGS. 7A to 7C are views schematically showing an example ofsuch specific subject images (face images) related to the subjectdetection processing.

Note that the subject detection processing, which will be describedbelow, is processing to be executed under the control of the centralcontrol section 11 in the case where a subject detection mode is setbased on the predetermined operation of the operation input unit 10 bythe user.

Moreover, in the following description, it is assumed to use such aframe image F in which face images as the specific subject images areincluded in the frame image F.

As shown in FIG. 2, first, the central control section 11 sequentiallystores such live view display-use image data of the frame images F,which are sequentially generated by the image data generation section 3by the image capturing of the subjects by the imaging section 1, in thememory 4, and allows the memory 4 to temporarily memorize the image data(Step S1).

Subsequently, the image obtaining unit 5 a of the image processingsection 5 obtains the live view display-use image data of the one frameimage Fn, which serves as a processing target, at the predeterminedtiming corresponding to the display frame rate by the display section 9(Step S2).

Next, the reduced image generation unit 5 b reduces the pixels in therespective horizontal and vertical directions in the image data of theone frame image Fn, which is obtained by the image obtaining unit 5 a,at every predetermined ratio (for example, by 0.9 time), and therebysequentially generates the reduced image data R (R1 to R5 . . . ) inwhich the resolution is reduced step by step (Step S3; refer to FIG. 4Ato FIG. 4F).

Subsequently, the tentative candidate detection unit c1 of thesimilarity degree evaluation value calculation unit 5 c generates theplurality of discrimination target regions C with the predetermined size(for example, 24×24 pixels) from each of the plurality of reduced imagedata R . . . (Step S4), and thereafter, calculates the evaluation valuerelated to the similarity degree between the image data of each of thediscrimination target regions C and the image data of the predeterminedreference image (Step S5). Specifically, the tentative candidatedetection unit c1 calculates the evaluation value in accordance with thediscrimination results of the sub-discriminators for the image data ofeach of the discrimination target regions C, for example, by using theadaboost output calculation (refer to FIG. 5 and FIG. 6).

Then, the tentative candidate detection unit c1 detects thediscrimination target region C, in which the calculated evaluation valueis larger than the predetermined value (for example, 0 “zero”), as thetentative candidate region of the specific subject image (Step S6).Here, a state where the tentative detection frame Wa is superimposed oneach of the tentative candidate regions (for example, face regions) ofthe plurality of specific subject images discriminated in each of thereduced image data R as shown in FIG. 7A is schematically shown. Forexample, with regard to a person who has a face with a larger area withrespect to the whole of the image, such as a person present on a frontside in a group picture, or to a person who faces to the front, it ismade possible to discriminate a face region thereof even in smallerreduced image data R in the case where the reduced image data R aresequentially generated. That is to say, with regard to the person whohas the face with a large area with respect to the whole of the image orfaces to the front, it becomes relatively easy to discriminate the faceregion thereof in comparison with a person who has a face with a smallerarea faces to the side. In such a way, on the face region concerned, alarger number of the tentative detection frames Wa are superimposed.

Note that the evaluation value of each discrimination target region Cdetected as the tentative candidate region is temporarily stored in therelated information storage unit 4 a of the memory 4.

Next, based on the coordinate positions in the XY plane spaces, whichare of the tentative candidate regions of the specific subject image,the tentative candidate regions being detected from the plurality ofreduced image data R . . . of the respective frame images F, on thesizes thereof, and on the like, the candidate region specifying unit c2of the similarity degree evaluation value calculation unit 5 cintegrates these tentative candidate regions with one another, andspecifies the candidate regions A of the specific image in therespective frame images F (Step S7; refer to FIG. 7A). Subsequently, thecandidate region specifying unit c2 performs the predeterminedarithmetic operation while taking as references the evaluation values ofthe plurality of tentative candidate regions, and calculates theevaluation value of the candidate region A of the specific subject image(Step S8).

Note that the calculated evaluation value of the candidate region A ofthe specific subject image is temporarily stored in the relatedinformation storage unit 4 a of the memory 4.

Then, the first similarity degree determination unit 5 d determineswhether or not the evaluation value of the candidate region A of thespecific subject image in the one frame image Fn, the evaluation valuebeing calculated by the similarity degree evaluation value calculationunit 5 c, is equal to or more than the first threshold value (Step S9).

Here, it is determined that the evaluation value of the candidate regionA of the specific subject image is equal to or more than the firstthreshold value (Steep S9; YES), the image region specifying unit 5 hspecifies the candidate region A (for example, the candidate regions A1to A4 or the like) of the specific subject image concerned as the imageregion D of the specific subject image (Step S10; refer to FIG. 7A andFIG. 7C).

Meanwhile, in the case where it is determined in Step S9 that theevaluation value of the candidate region A (for example, the candidateregion A5 or the like) is not equal to or more than the first thresholdvalue (Step S9; NO), the second similarity degree determination unit 5 edetermines whether or not the evaluation value of the candidate region Aof the specific subject image concerned is equal to or more than thesecond threshold value (Step S11).

Here, when it is determined that the evaluation value of the candidateregion A of the specific subject image is not equal to or more than thesecond threshold value (Step S11; YES), the related informationobtaining unit 5 f specifies the region, in which it is determined thatthe evaluation value is equal to or more than the second thresholdvalue, in the other frame image Fm generated the predetermined number offrames (for example, one frame or the like) before from the one frameimage Fn (Step S12; refer to FIG. 7B), and obtains the similarity degreeinformation of the region B concerned from the related informationstorage unit 4 a of the memory 4 (Step S13). Specifically, the relatedinformation obtaining unit 5 f obtains, as the similarity degreeinformation, the evaluation degree of the region B, the number of thetentative candidate regions used for specifying the candidate region Aof the specific subject image, which corresponds to the regionconcerned, and the like from the related information storage unit 4 a.

Then, the subject determination unit 5 g determines whether or not theevaluation value of the region B, which is the similarity degreeinformation obtained by the related information obtaining unit 5 f, isequal to or more than the predetermined determination value, and whetheror not the number of the tentative candidate regions, which is thesimilarity degree information obtained by the related informationobtaining unit 5 f, is equal to or more than the predetermineddetermination value (Step S14).

Here, when it is determined that the evaluation value of the region B isequal to or more than the predetermined determination value, and thatthe number of the tentative candidate regions is equal to or more thanthe predetermined determination value (Step S14; YES), the image regionspecifying unit 5 h specifies the candidate region A of the specificsubject image, which corresponds to the region B concerned, that is,specifies the candidate region A of the specific subject image, in whichthe evaluation value is equal to or more than the second thresholdvalue, as the image region D of the specific subject image (Step S15;refer to FIG. 7C).

Then, after Step S10 and Step S15, the discrimination informationsetting section 6 sets the detection frame Wb while superimposing thedetection frame Wb concerned on the edge portion of the image region Dof the specific subject image, which is specified in the one frame imageFn by the image region specifying unit 5 h, and the display controlsection 8 allows the display section 9 to perform the live view displayfor the one frame image Fn concerned.

Thereafter, the central control section 11 determines whether or not anending instruction of the subject detection processing is inputtedthereto (Step S16). Specifically, the central control section 11determines whether or not the ending instruction of the subjectdetection processing is inputted thereto, for example, in response towhether or not the image capturing instruction of the subject isinputted based on a predetermined operation (for example, a full pressoperation and the like) for the shutter button of the operation inputsection 10 by the user, or in response to whether or not a modedifferent from the subject detection mode is set based on apredetermined operation for the selection decision button.

Moreover, also in the case where it is determined in Step S11 that theevaluation value of the candidate region A of the specific subject imageis not equal to or more than the second threshold value (Step S11; NO),or where it is determined in Step S14 that the evaluation value of theregion B is not equal to or more than the predetermined determinationvalue, or that the number of the tentative candidate regions is notequal to or more than the predetermined determination value (Step S14;NO), the central control section 11 shifts the processing to Step S16,and determines whether or not the ending instruction of the subjectdetection processing is inputted (Step S16).

When it is determined in Step S16 that the ending instruction of thesubject detection processing is not inputted (Step S16; NO), the centralcontrol section 11 shifts the processing to Step S2, and the imageobtaining unit 5 a obtains live view display-use image data of a one newframe image from the memory 4 (Step S2). Thereafter, the central controlsection 11 executes the respective pieces of processing of Step S3 andafter in a similar way to the above.

Meanwhile, when it is determined in Step S16 that the ending instructionof the subject detection processing is inputted (Step S16; YES), thecentral control section 11 ends the subject detection processing.

As described above, in accordance with the image capturing apparatus 100of this embodiment, when it is determined that the evaluation value(similarity degree) between the image of the candidate region A of thespecific subject image (for example, the face image and the like) in theobtained one frame image Fn (for example, the live view display-useimage generated from the captured image, and the like) and thepredetermined reference image serving as the determination criteria ofthe specific subject image concerned is equal to or more than the firstthreshold value, the candidate region A of the specific subject image isspecified as the image region D of the specific subject image.Accordingly, the first threshold value, which is more severe, can beset, whereby it can be determined whether or not the candidate region Aof the specific subject image is the image region D of the specificsubject image in the one frame image Fn, and the erroneous detection ofthe specific subject concerned can be reduced.

Meanwhile, in the case where it is determined that the evaluation valueof the candidate region A of the specific subject image is not equal toor more than the first threshold value, it is determined whether or notthe evaluation value concerned is equal to or more than the secondthreshold value smaller than the first threshold value. When theevaluation value of the candidate region A of the specific subject imageis equal to or more than the second threshold value, the similaritydegree information is obtained, which is related to the similaritydegree between the predetermined reference image and the image of theregion B, which corresponds to the candidate region A of the specificsubject image, in the other frame image Fm obtained the predeterminednumber of frames before from the one frame image Fn. Then, based on theobtained similarity degree information, it is determined whether or notthe candidate region A of the specific subject image, which correspondsto the region B in the one frame image Fn, should be specified as theimage region D of the specific subject image. Accordingly, if theevaluation value of the candidate region A of the specific subject imageis equal to or more than the second threshold value even if theevaluation value concerned is less than the first threshold value, thenit can be specified whether or not the candidate region A belongs to thespecific subject image based on the similarity degree information of theregion B, which corresponds to the candidate region A of the specificsubject image concerned, in the other frame image Fm, and the loweringof the detection rate of the specific subject can be suppressed.

As described above, even if the detection rate of the specific subjectto be detected by using the more severe first threshold value is loweredby the fact that the threshold value concerned is set, if the evaluationvalue of the candidate region A of the specific subject image is equalto or more than the second threshold value, then there is room where thecandidate region A is specified as the image region D of the specificsubject image. Accordingly, the reduction of the erroneous detection ofthe specific subject can be achieved without lowering the detection rateof the specific subject concerned.

Moreover, in the case where it is determined that the evaluation valuebetween the image of the region B and the predetermined reference imageis equal to or more than the predetermined determination value, theimage region specifying unit 5 h specifies the candidate region A of thespecific image, which corresponds to the region B concerned, as theimage region D of the specific subject image. Therefore, the imageregion specifying unit 5 h can specify the candidate region A of thespecific subject image, which corresponds to the region B in which theevaluation value in the other frame image Fm is equal to or more thanthe predetermined determination value, as the image region D of thespecific subject image, and can appropriately suppress the lowering ofthe detection rate of the specific subject.

Moreover, in the case where it is determined that the number of thetentative candidate regions of the specific subject image, whichcorrespond to the region B and are detected from the plurality ofreduced images at the substantially equal positions, are equal to ormore than the predetermined determination value, the image regionspecifying unit 5 h specifies the candidate region A of the specificimage, which corresponds to the region B concerned, as the image regionD of the specific subject image. Therefore, the image region specifyingunit 5 h can specify the candidate region A of the specific subjectimage, which corresponds to the region B in which the number of thetentative candidate regions of the specific subject image in the otherframe image Fm is equal to or more than the predetermined determinationvalue, as the image region D of the specific subject image, and canappropriately suppress the lowering of the detection rate of thespecific subject.

Moreover, the candidate region A of the specific subject image in theframe F concerned is specified based on the tentative candidate regionsof the specific subject image, which are detected from the plurality ofreduced image data R . . . at the substantially equal positions.Accordingly, increase and decrease of the number of the tentativecandidate regions can be adjusted by increasing and decreasing thenumber of the reduced image data R to be generated, and it is possibleto shorten a time required for the change of the discrimination accuracyof the candidate region A of the specific subject image in the image orthe detection processing for the image region D of the specific subjectimage, and so on.

Note that the present invention is not limited to the above-describedembodiment, and may be improved and changed in design in various wayswithin the scope without departing from the spirit of the presentinvention.

For example, in the subject detection processing (refer to FIG. 2), thehuman face image is detected as the specific subject image in the frameimage F; however, this is merely an example, and the specific subjectimage according to the present invention is not limited to this, and ischangeable appropriately and arbitrarily.

Moreover, in the above-described embodiment, as the similarity degreeinformation, there are illustrated: the evaluation value related to thesimilarity degree between the image of the region B and thepredetermined reference image; the number of the tentative candidateregions of the specific subject image, which are detected from theplurality of reduced images at the substantially equal positions; andthe like; however, these are merely an example, and the similaritydegree information according to the present invention is not limited tothese, and is changeable appropriately and arbitrarily.

Moreover, in the above-described embodiment, the discriminationinformation setting section 6 is provided, and the discriminationinformation (for example, the detection frame Wb and the like) fordisplaying the specified image region D of the specific subject image ina discriminating manner is set; however, it is possible to appropriatelyand arbitrarily make a change as to whether or not to provide thediscrimination information setting unit 6, that is, as to whether or notto set the discrimination information for displaying the specificsubject image, which is specified by the image region specifying unit 5h, in a discriminating manner.

Moreover, the configuration of the image capturing apparatus 100, whichis illustrated in the above-described embodiment, is merely an example,and the configuration thereof according to the present is not limited tothis. For example, the image capturing apparatus 100 is illustrated asthe subject determination apparatus; however, the subject determinationapparatus according to the present invention is not limited to this.That is to say, though the images generated from the captured image areillustrated as such frame images F that serves as the processingtargets, the images concerned just need to be frame images, which are tobe sequentially obtained by predetermined obtaining means, in the casewhere the subject determination apparatus is composed of an apparatusdifferent from the image capturing apparatus 100.

Moreover, the similarity degree evaluation value calculation unit 5 c isprovided; however, it is not always necessary to provide the similaritydegree evaluation value calculation unit 5 c concerned. For example,predetermined obtaining means may obtain the evaluation values of therespective candidate regions of the specific subject image together withthe frame images F.

In addition, in the above-described embodiment, a configuration isadopted, in which functions as the obtaining means, the first similaritydegree determining means, the second similarity degree determiningmeans, the information obtaining means, and the subject determiningmeans are realized in such a manner that the image obtaining unit 5 a,first similarity degree determination unit 5 d, second similarity degreedetermination unit 5 e, related information obtaining unit 5 f andsubject determination unit 5 g of the image processing section 5 aredriven under the control of the central control section 11; however,such a configuration according to the present invention is not limitedto this. A configuration in which a predetermined program and the likeare executed by the CPU of the central control section 11 may beadopted.

That is to say, in a program memory (not shown) that memorizes programstherein, a program is memorized, which includes an obtaining processingroutine, a first similarity degree determination processing routine, asecond similarity degree determination processing routine, aninformation obtaining processing routine, and a subject determinationprocessing routine. Then, by the obtaining processing routine, the CPUof the central control section 11 may be allowed to function as theobtaining means for sequentially obtaining the frame images F. Moreover,by the first similarity degree determination processing routine, the CPUof the central control section 11 may be allowed to function as thefirst similarity degree determining means for determining whether or notthe similarity degree between the predetermined reference image and theimage of the candidate region A of the specific subject in the frameimage Fn obtained by the obtaining means is equal to or more than thefirst threshold value. Furthermore, by the second similarity degreedetermination processing routine, the CPU of the central control section11 may be allowed to function as the second similarity degreedetermining means for determining whether or not the similarity degreeis equal to or more than the second threshold value smaller than thefirst threshold value in the case where it is determined that thesimilarity degree concerned is not equal to or more than the firstthreshold value by the first similarity degree determining means.Moreover, by the information obtaining processing routine, the CPU ofthe central control unit 11 may be allowed to function as theinformation obtaining means for obtaining the information related to thesimilarity degree between the predetermined reference image and theimage of the region B, which corresponds to the candidate region A inthe frame image Fm obtained the predetermined number of frames beforefrom the frame image Fn, in the case where it is determined that thesimilarity degree of the candidate region A is equal to or more than thesecond threshold value by the second similarity degree determiningmeans. Furthermore, by the subject determination processing routine, theCPU of the central controls section 11 may be allowed to function as thesubject determining means for determining whether or not the candidateregion A is the image region D of the specific subject image based onthe information obtained by the information obtaining means.

In a similar way, such a configuration may be adopted, in which subjectspecifying means, the first memorization controlling means, imagereducing means, detecting means, second memorization controlling means,and candidate specifying means are also realized in such a manner that apredetermined program and the like are executed by the CPU of thecentral control section 11.

Moreover, as computer-readable mediums which store therein the programsfor executing the above-described respective pieces of processing, it isalso possible to apply a non-volatile memory such as a flash memory, anda portable recording medium such as a CD-ROM as well as a ROM, a harddisk and the like. Moreover, as a medium that provides data of theprograms through a predetermined communication line, a carrier wave isalso applied.

Some of the embodiments of the present invention have been described;however, the scope of the present invention is not limited to theabove-mentioned embodiments, and incorporates the scope of theinvention, which is described in the scope of claims, and incorporatesequilibrium ranges thereof.

The entire disclosure of Japanese Patent Application No. 2012-024947filed on Feb. 8, 2012 including description, claims, drawings, andabstract are incorporated herein by reference in its entirety.

Although various exemplary embodiments have been shown and described,the invention is not limited to the embodiments shown. Therefore, thescope of the invention is intended to be limited solely by the scope ofthe claims that follow.

What is claimed is:
 1. A subject determination apparatus comprising: animage obtaining unit that sequentially obtains frame images; a firstobtaining unit that obtains, from a first region in a first frame imagefrom among the obtained frame images, a first degree of similarity to aspecific subject image; a first similarity degree determination unitthat determines whether or not the first degree of similarity is equalto or more than a first threshold value; a first subject determinationunit that determines that the first region is a region of the specificsubject image, when the first similarity degree determination unitdetermines that the first degree of similarity is equal to or more thanthe first threshold value; a second obtaining unit that obtains, from asecond region in a second frame image from among the obtained frameimages, a second degree of similarity to the specific subject image, thesecond frame image being an image obtained before the first frame imageby the image obtaining unit; a second similarity degree determinationunit that determines whether or not the second degree of similarity isequal to or more than a second threshold value; and a second subjectdetermination unit that determines that the first region is the regionof the specific subject image, when (i) the first similarity degreedetermination unit determines that the first degree of similarity is notequal to or more than the first threshold value, and (ii) the secondsimilarity degree determination unit determines that the second degreeof similarity is equal to or more than the second threshold value. 2.The subject determination apparatus according to claim 1, furthercomprising: an image reduction unit that sequentially reduces the frameimages obtained by the image obtaining unit with a predetermined ratioto generate reduced images; a detection unit that detects tentativecandidate regions of the specific subject image from the plurality ofthe reduced images, the tentative candidate regions being located atcorresponding positions in the plurality of the reduced images; and anumber information obtaining unit that obtains a number of the tentativecandidate regions, the tentative candidate regions being detected fromthe plurality of the reduced images, wherein the second subjectdetermination unit determines that the first region is the region of thespecific subject image based on the number of the tentative candidateregions obtained by the number information obtaining unit.
 3. Thesubject determination apparatus according to claim 1, wherein thespecific subject image includes a face image.
 4. The subjectdetermination apparatus according to claim 1, wherein the frame imagesare images generated from a captured image.
 5. The subject determinationapparatus according to claim 1, wherein a position of the second regionin the second frame image is a substantially same position as a positionof the first region in the first frame image.
 6. The subjectdetermination apparatus according to claim 1, further comprising: athird similarity degree determination unit that determines whether ornot the first degree of similarity is equal to or more than a thirdthreshold value, wherein the third threshold value is smaller than thefirst threshold value, when the first similarity degree determinationunit determines that the first degree of similarity is not equal to ormore than the first threshold value, wherein the second obtaining unitobtains the second degree of similarity, when the third similaritydegree determination unit determines that the first degree of similarityis not equal to or more than the third threshold value.
 7. The subjectdetermination apparatus according to claim 6, wherein the thirdthreshold value is the second threshold value.
 8. A method ofdetermining a subject by using a subject determination apparatus, themethod comprising: sequentially obtaining frame images; obtaining, froma first region in a first frame image from among the obtained frameimages, a first degree of similarity to a specific subject image;determining whether or not the first degree of similarity is equal to ormore than a first threshold value; determining that the first region isa region of the specific subject image, when it is determined that thefirst degree of similarity is equal to or more than the first thresholdvalue; when it is determined that the first degree of similarity is notequal to or more than the first threshold value, (i) obtaining, from asecond region in a second frame image from among the obtained frameimages, a second degree of similarity to the specific subject image, thesecond frame image being an image obtained before the first frame image,and (ii) determining whether or not the second degree of similarity isequal to or more than a second threshold value; and determining that thefirst region is the region of the specific subject image, when it isdetermined that the first degree of similarity is not equal to or morethan the first threshold value and that the second degree of similarityis equal to or more than the second threshold value.
 9. A non-transitorycomputer-readable recording medium having a program stored thereon thatis executable to control a computer to function as units comprising: afirst obtaining unit that obtains, from a first region in a first frameimage from among sequentially obtained frame images, a first degree ofsimilarity to a specific subject image; a first similarity degreedetermination unit that determines whether or not the first degree ofsimilarity is equal to or more than a first threshold value; a firstsubject determination unit that determines that the first region is aregion of the specific subject image, when the first similarity degreedetermination unit determines that the first degree of similarity isequal to or more than the first threshold value; a second obtaining unitthat obtains, from a second region in a second frame image from amongthe sequentially obtained frame images, a second degree of similarity tothe specific subject image, the second frame image being an imageobtained before the first frame image from among the sequentiallyobtained frame images; a second similarity degree determination unitthat determines whether or not the second degree of similarity is equalto or more than a second threshold value; and a second subjectdetermination unit that determines that the first region is the regionof the specific subject image, when (i) the first similarity degreedetermination unit determines that the first degree of similarity is notequal to or more than the first threshold value, and (ii) the secondsimilarity degree determination unit determines that the second degreeof similarity is equal to or more than the second threshold value.